有效的视觉在延迟预算下的精度最大化。这些作品一次评估脱机准确性，一次是一张图像。但是，诸如自动驾驶之类的实时视觉应用在流媒体设置中运行，在这些设置中，地面真相在推理开始和终点之间会发生变化。这会导致明显的准确性下降。因此，最近提出的一项旨在最大程度地提高流媒体设置准确性的工作。在本文中，我们建议在每个环境环境中最大化流的准确性。我们认为场景难度会影响初始（离线）精度差异，而场景中的障碍物位移会影响后续的准确性降解。我们的方法章鱼使用这些方案属性来选择在测试时最大化流量准确性的配置。我们的方法将跟踪性能（S-MOTA）提高了7.4％，而常规静态方法则提高了。此外，使用我们的方法提高性能，而不是离线准确性的进步，而不是代替而不是进步。

人们普遍认为，在传输学习中，包括更多的预训练数据可以转化为更好的性能。但是，最近的证据表明，从源数据集中删除数据实际上也可以提供帮助。在这项工作中，我们仔细研究了源数据集在转移学习中的作用，并提出了探索其对下游性能的影响的框架。我们的框架产生了新的功能，例如精确转移学习脆弱性以及检测诸如数据渗漏等病理和源数据集中存在误导示例之类的病理。特别是，我们证明，消除通过框架确定的有害数据点可改善来自ImageNet的转移学习绩效，以了解各种目标任务。代码可从https://github.com/madrylab/data-transfer获得

使用转移学习将预先训练的“源模型”调整为下游“目标任务”可以大大提高性能，而似乎没有缺点。在这项工作中，我们证明毕竟可能存在一个缺点：偏差转移或源模型偏见的趋势，即使将模型调整为目标类别后，也可以持续存在。通过合成和自然实验的组合，我们表明偏差转移（a）是在现实设置中（例如，在图像网或其他标准数据集上进行预训练时）以及（b）即使明确数据也可能发生（b） - 偏见。随着转移学习的模型越来越多地在现实世界中部署，我们的工作突出了理解预训练源模型的局限性的重要性。代码可从https://github.com/madrylab/bias-transfer获得

语言模型既展示了定量的改进，又展示了新的定性功能，随着规模的增加。尽管它们具有潜在的变革性影响，但这些新能力的特征却很差。为了为未来的研究提供信息，为破坏性的新模型能力做准备，并改善社会有害的效果，至关重要的是，我们必须了解目前和近乎未来的能力和语言模型的局限性。为了应对这一挑战，我们介绍了超越模仿游戏基准（Big Bench）。 Big Bench目前由204个任务组成，由132家机构的442位作者贡献。任务主题是多样的，从语言学，儿童发展，数学，常识性推理，生物学，物理学，社会偏见，软件开发等等。 Big-Bench专注于被认为超出当前语言模型的功能的任务。我们评估了OpenAI的GPT型号，Google内部密集变压器体系结构和大型基础上的开关稀疏变压器的行为，跨越了数百万到数十亿个参数。此外，一个人类专家评估者团队执行了所有任务，以提供强大的基准。研究结果包括：模型性能和校准都随规模改善，但绝对的术语（以及与评估者的性能相比）；在模型类中的性能非常相似，尽管带有稀疏性。逐渐和预测的任务通常涉及大量知识或记忆成分，而在临界规模上表现出“突破性”行为的任务通常涉及多个步骤或组成部分或脆性指标；社交偏见通常会随着含糊不清的环境而随着规模而增加，但这可以通过提示来改善。

缺失或缺乏输入功能，是许多模型调试工具的基础概念。但是，在计算机视觉中，不能简单地从图像中删除像素。因此，一种倾向于诉诸启发式方法，例如涂黑像素，这反过来又可能引入调试过程中的偏见。我们研究了这样的偏见，特别是展示了基于变压器的架构如何使遗失性更自然地实施，哪些侧架来侧翼这些问题并提高了实践中模型调试的可靠性。我们的代码可从https://github.com/madrylab/missingness获得

分析深神经网络对输入扰动的最坏情况的性能等于解决一个大规模的非凸优化问题，过去的几项工作提出了凸出的放松作为有希望的替代方案。但是，即使对于合理的神经网络，这些放松也无法处理，因此必须在实践中被较弱的放松所取代。在这项工作中，我们提出了一种新型的操作员分裂方法，该方法可以将问题直接解决至高精度的凸松弛，从而将其拆分为经常具有分析溶液的较小的子问题。该方法是模块化的，范围为非常大的问题实例，并损害了与GPU加速的快速并行化的操作。我们展示了我们在图像分类和强化学习设置以及神经网络动力学系统的可及性分析中界定大型卷积网络最差的方法的方法。

It is common practice in deep learning to use overparameterized networks and train for as long as possible; there are numerous studies that show, both theoretically and empirically, that such practices surprisingly do not unduly harm the generalization performance of the classifier. In this paper, we empirically study this phenomenon in the setting of adversarially trained deep networks, which are trained to minimize the loss under worst-case adversarial perturbations. We find that overfitting to the training set does in fact harm robust performance to a very large degree in adversarially robust training across multiple datasets (SVHN, CIFAR-10, CIFAR-100, and ImageNet) and perturbation models ( ∞ and 2 ). Based upon this observed effect, we show that the performance gains of virtually all recent algorithmic improvements upon adversarial training can be matched by simply using early stopping. We also show that effects such as the double descent curve do still occur in adversarially trained models, yet fail to explain the observed overfitting. Finally, we study several classical and modern deep learning remedies for overfitting, including regularization and data augmentation, and find that no approach in isolation improves significantly upon the gains achieved by early stopping. All code for reproducing the experiments as well as pretrained model weights and training logs can be found at https://github.com/ locuslab/robust_overfitting.

Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with = 8/255 in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at = 2/255 in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https://github.com/locuslab/fast_adversarial.

We propose a method to learn deep ReLU-based classifiers that are provably robust against normbounded adversarial perturbations on the training data. For previously unseen examples, the approach is guaranteed to detect all adversarial examples, though it may flag some non-adversarial examples as well. The basic idea is to consider a convex outer approximation of the set of activations reachable through a norm-bounded perturbation, and we develop a robust optimization procedure that minimizes the worst case loss over this outer region (via a linear program). Crucially, we show that the dual problem to this linear program can be represented itself as a deep network similar to the backpropagation network, leading to very efficient optimization approaches that produce guaranteed bounds on the robust loss. The end result is that by executing a few more forward and backward passes through a slightly modified version of the original network (though possibly with much larger batch sizes), we can learn a classifier that is provably robust to any norm-bounded adversarial attack. We illustrate the approach on a number of tasks to train classifiers with robust adversarial guarantees (e.g. for MNIST, we produce a convolutional classifier that provably has less than 5.8% test error for any adversarial attack with bounded ∞ norm less than = 0.1), and code for all experiments is available at http://github.com/ locuslab/convex_adversarial.

Non-linear state-space models, also known as general hidden Markov models, are ubiquitous in statistical machine learning, being the most classical generative models for serial data and sequences in general. The particle-based, rapid incremental smoother PaRIS is a sequential Monte Carlo (SMC) technique allowing for efficient online approximation of expectations of additive functionals under the smoothing distribution in these models. Such expectations appear naturally in several learning contexts, such as likelihood estimation (MLE) and Markov score climbing (MSC). PARIS has linear computational complexity, limited memory requirements and comes with non-asymptotic bounds, convergence results and stability guarantees. Still, being based on self-normalised importance sampling, the PaRIS estimator is biased. Our first contribution is to design a novel additive smoothing algorithm, the Parisian particle Gibbs PPG sampler, which can be viewed as a PaRIS algorithm driven by conditional SMC moves, resulting in bias-reduced estimates of the targeted quantities. We substantiate the PPG algorithm with theoretical results, including new bounds on bias and variance as well as deviation inequalities. Our second contribution is to apply PPG in a learning framework, covering MLE and MSC as special examples. In this context, we establish, under standard assumptions, non-asymptotic bounds highlighting the value of bias reduction and the implicit Rao--Blackwellization of PPG. These are the first non-asymptotic results of this kind in this setting. We illustrate our theoretical results with numerical experiments supporting our claims.